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Abstract 


This study investigated the moderating roles of motivation, social engagement, and self- 
regulation in predicting academic achievement in grades 11-12 for 3,281 7th-9th graders. 
Standardized assessments of college readiness (ACT® Explore and the ACT® test) and student 
self-report measures of social emotional learning factors (ACT® Engage® Grades 6-9) were 
used in the study. The results showed that females in grades 7-9 scored higher than males for 
motivation and self-regulation. Motivation and social engagement moderated prior achievement 
in predicting later achievement in grades 11-12, while self-regulation moderated both gender 
and prior achievement in predicting later achievement. Specifically, among female students, 
effects were positive for females with higher prior achievement (PR 2 95) and negative for 
females with lower prior achievement (PR < 5) for self-regulation in predicting achievement in 


grades 11-12. 


Introduction 


Social emotional learning (SEL) factors, such 
as motivation, social engagement, and self- 
regulation, play key roles in classroom 
learning (Zins, Bloodworth, Weissberg, & 
Walberg, 2007). Motivation refers to self- 
regulatory mechanism to achieve academic 
success via certain personality characteristics 
and attitudes, such as Academic Discipline, 
Commitment to School, and Optimism. Social 
engagement refers to students’ perception of 
their family’s engagement, particularly family 
members’ attitudes regarding the value of 
education and their involvement in school 
activities, relationships with school personnel, 
and perception of school safety climate. Self- 
regulation refers to the degree to which 
students appropriately regulate and express 
their feelings and behaviors, as well as how 
they think about the consequences of 
behavior in school contexts. Students need to 


be highly motivated to learn well in school, to 
recognize and use the social supports that 
can facilitate their learning, and to regulate 
their behaviors and manage their feelings. 
From a broader perspective, SEL factors and 
prior academic achievement have been 
demonstrated to predict later achievement in 
both K-12 and higher education settings (e.g., 
Casillas et al., 2012; Poropat, 2009). 
However, it is unclear how SEL factors 
interact with prior academic achievement and 
gender when predicting subsequent 
academic achievement. 


In educational contexts, the Matthew Effect is 
one theory for explaining why the variability of 
achievement and ability increases over time, 
which leads to a widening gap between high- 
achieving and low-achieving students. (e.g., 
Ceci & Papierno, 2005). For example, 
Walberg and Tsai (1983) conducted a 
longitudinal study through National 
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Assessment of Educational Progress and 
found that prior educational background, 
current educational experience, and 
motivation significantly predict academic 
achievement individually and together, while 
the cumulating advantageous effects of these 
three factors were evidenced in the prediction 
models. Stated differently, students 
possessing better prior educational 
background tend to have better educational 
experience and be motivated to learn. 
Furthermore, the cumulative advantages of 
these factors show predictive power for later 
academic outcomes. Consequently, the 
achievement gap is widening over time due to 
students’ increasing differences in 
educational backgrounds. Although middle 
school students are not directly represented 
in past studies, it seems reasonable to 
hypothesize that high-achieving middle 
school students may benefit more over time 
in achievement from particular SEL factors 
than low-achieving students. 


The purpose of this study is to investigate the 
roles played by SEL factors and gender in 
predicting academic achievement in grades 
11-12 among 7th-9th-grade students. 
Specifically, the differential effects of SEL 
factors for students who differed in academic 
achievement and gender were examined. 
The research questions of this study were: 


1. Do male and female 7th-9th graders differ 
on SEL characteristics? 


2. Do students’ SEL factors interact with 
gender and prior academic achievement 
in the prediction of achievement in grades 
11-12? Does the Matthew Effect take 
place as SEL factors interact with prior 
academic achievement? 


Method 


Database 


To assess the full range of students’ SEL 
factors and academic achievement, an ACT 
longitudinal database was used. This 
database was designed for the purpose of 
development and validation of Engage 
Grades 6-9, a self-report measure of 
students’ SEL factors related to academic 
performance and persistence. The students 
in the database were initially recruited from 
24 middle schools in 13 districts showing a 
broad range of student demographic and 
achievement variability, had used the ACT 
Explore academic achievement test, and 
agreed to provide follow-up data as the 
students progressed through high school 
(ACT, 2011). 


Participants 


Students were included based on three 
criteria: (a) students took an assessment of 
SEL factors (ACT Engage Grades 6-9) while 
in grade 7, 8, or 9; (b) students took college 
readiness assessments in grade 7, 8, or 9 
(ACT Explore) and grade 11 or 12 (the ACT 
test), and (c) students’ responses in the SEL 
assessment were not flagged for insufficient 
item response variability or response 
inconsistency. The sample consisted of 
3,281 students, 3% of whom took the 
assessment of SEL factors in 7th grade, 59% 
in 8th grade, and 38% in 9th 

grade. Approximately half of the students 
were female (55%) and predominantly White 
(43%), with the remaining racial/ethnic 
breakdown being: 22% Hispanic/Latino, 23% 
African American, 5% Multiracial, 2% 
American Indian/Alaskan Native, 2% Asian 
American/Pacific Islander, and 3% of another 
race/ethnicity or preferred not to respond. 
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Measures 


SEL. ACT Engage Grades 6-9 includes nine 
scales using 6-point Likert-type items (1 = 
Disagree a Lot and 6 = Agree a Lot) and one 
scale (Orderly Conduct) using a binary 
response format (1 = Yes and 0 = No). The 
10 scales show moderate to high internal 
reliabilities (a = .81 - .90, Mdn = .87; ACT, 
2011). The SEL scales were grouped into 
three higher-order SEL factors used in the 
study: Motivation (consisting of Academic 
Discipline, Commitment to School, and 
Optimism), Social Engagement (consisting of 
Family Attitude toward Education and Family 
Involvement, Relationships with School 
Personnel, and School Safety Climate), and 
Self-Regulation (consisting of Managing 
Feelings, Orderly Conduct, and Thinking 
before Acting). 


Academic achievement. Composite 
scores from ACT Explore and the ACT were 
used as estimates of students’ overall 
academic achievement at two points in time. 
Although using Composite scores eliminates 
the ability to explore relationships for 
particular subject areas (e.g., English, 
mathematics, reading, and science), they 
provide a general proxy on academic 
achievement. Scale scores for Explore and 
the ACT are comparable and could be 
measured on the same scale to facilitate 
longitudinal comparisons (ACT, 2013, 2014). 


Data Analyses 


Given unequal item numbers and different 
item formats of Engage scales, raw mean 
scale scores were first linearly transformed to 
new scale scores ranging from 10 to 60. After 
obtaining Engage scale scores on the 
common score range, the unit weights 


method was applied to sum the scores to 
form aggregate scores for three SEL factors: 
motivation, social engagement, and self- 
regulation. The aggregate SEL scores and 
students’ Explore and ACT Composite scores 
were transformed to z-scores for the following 
analyses. The dichotomous gender variable 
(males = 1 and females = 0) was not 
standardized. Further, those standardized 
SEL scores that were three standard 
deviations below the mean were considered 
outliers and removed from further analyses. 
As a result, 1.3% (n = 43), .5% (n = 16), and 
.1% (n = 3) of motivation, social engagement, 
and self-regulation scores, respectively, were 
removed. 


In response to research question 1, 
comparisons between male and female 
students in grades 7-9 were conducted using 
independent-sample Student's t-tests for 
each SEL factor. The effect size (Cohen's d) 
was also estimated. In response to the 
research question 2, multiple linear 
regression models were run to test the 
hypotheses that each SEL factor moderates 
gender and prior academic achievement in 
the prediction of later achievement. Post hoc 
probing techniques were used if significant 
interaction terms were found. 


Results 


Gender Differences in SEL 
Factors 


Table 1 shows that female students tended to 
outperform male students on motivation and 
self-regulation. The effect size for gender 
differences were small for motivation (d = 
.20), social engagement (d = .03), and self- 
regulation (d = .24). 
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Table 1. Comparison of SEL Factors across Male and Female Students in Grades 7-9 


SEL Females Males 

Motivation 52.34 51.31 

Female (n = 1,779) (5.10) (5.31) 
Male (n = 1,431) 

Social Engagement 45.41 45.23 

Female (n = 1,786) (6.37) (6.07) 
Male (n = 1,450) 

Self-Regulation 42.98 40.71 

Female (n = 1,793) (9.45) (9.01) 


Male (n = 1,455) 


Moderating Effects of SEL 
Factors 


Motivation. Table 2 and Figure 1 show that 
the relationships between prior and later 
achievement as moderated by motivation. 
The two-way interaction terms between prior 
achievement and motivation in predicting 
later achievement were statistically significant 
(B = .040, p < .001). For high-achieving 
students (i.e., scores in prior academic 
achievement are PR 2 95), the slope is 


Table 2. Moderating Effects of Motivation 
I Coxe [=1 i 


B SE 


Intercept -.059"" 013 
Gender 129” .020 
Explore 845" 014 
Gender x Explore -.024 .020 
Motivation 

Gender x Motivation 

Explore x Motivation 

Model R? .6896 .6926 
AR? 


n=3,210. ™ p< .001 


t (oli Cohen’s d 
5.577" 3208 20 
.80 3234 .03 
6.91 *** 3246 24 


slightly positive, whereas for low-achieving 
students (i.e., scores in prior academic 
achievement are PR < 5), the slope is slightly 
negative. That is, as motivation increased, 
regardless of gender, the difference of later 
academic achievement generally widened. 
The results followed the pattern we expected. 
Additionally, motivation provided a slight 
contribution beyond gender and prior 
achievement scores in predicting later 
achievement (AR? = .003). 


WY KoYe (=) 4 


-.072" 013 
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Figure 1. Moderating Effects of Motivation and Later Academic Achievement at Low, Average, 
and High Scores 
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Social engagement. Table 3 and Figure 2 show the relationships between prior and later 
achievement as moderated by social engagement in predicting later achievement (B = .032, p < 
.01), and the two-way moderation effects are illustrated in Figure 2. In the prediction of later 
achievement, the findings suggest that high-achieving students benefit more from higher social 
control, whereas low-achieving students do not; this is regardless of gender. Similar to findings in 
motivation, as social engagement increased, high achieving students experienced more growth in 
general than low-achieving students. Overall, social engagement provided incremental 
contribution beyond gender and prior achievement scores in predicting later achievement (AR? = 
.005). 


Table 3. Moderating Effects of Social Engagement 


WM Coke (=1 i LM Coke (1 7 
B SE B SE 

Intercept -.058"" 013 -.065" 013 
Gender 129" .020 ‘a3 .020 
Explore 844" .014 829" .014 
Gender x Explore -.026 .020 -.015 .020 
Social Engagement 081" 013 
Gender x Social Engagement -.051° .020 
Explore x Social Engagement 032" .010 
Model R? .6895 .6944 

AR2 .0049 


N = 3,236. ° p < .05 (two-tailed). * p < .01 (two-tailed). ™ p < .001 
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Figure 2. Moderating Effects of Social Engagement and Later Academic Achievement at Low, 
Average, and High Scores 
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Self-regulation. Table 4 and Figure 3 show the relationships between prior and later 
achievement as moderated by gender and self-regulation in predicting later achievement (B = - 
.061, p < .01). Self-regulation provided larger incremental contribution beyond gender and prior 
achievement scores in predicting later achievement (AR? = .0076). Table 5 shows that the slope 
of high-achieving female students significantly differed from high-achieving male students in 
predicting later achievement. In addition, the slopes of high- and low-achieving female students 
were significantly different. That is, as self-regulation increased, effects of prior achievement on 
later achievement increased for female students. To that end, among female students, the 
Matthew Effect was evident. 


Table 4. Moderating Effects of Self-Regulation 


WM Coke (=1 i 
SE 
Intercept -.057"" 013 -.085"" 013 
Gender 129" .020 .160™" .020 
Explore 844" 013 801" 014 
Gender x Explore -.027 .020 .005 .020 
Self-regulation .086™ 013 
Gender x Self-regulation -.043° .020 
Explore x Self-regulation .078™" 014 
Gender x Explore x Self-regulation -.061~ .020 
Model R? .6896 6972 
AR? .0076 


N = 3,248. *p<.05,"p<.01, “p<.001 
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Figure 3. Moderating Effects of Self-Regulation, Gender and Later Academic Achievement at 
Low, Average, and High Scores 
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Table 5. Tests of Simple Slopes of Later Academic Achievement on Self-regulation 


Slope Differences SEcontrast Bonferroni’s p 
High-Achieving Male vs. High-Achieving Female -.16 04 -3.68° < .001 
Low-Achieving Male vs. Low-Achieving Female 05 04 1.31 .048 
High-Achieving Male vs. Low-Achieving Male .06 05 1.14 .063 
High-Achieving Female vs. Low-Achieving Female 26 05 5.78 < .001 


N = 3,248. Slope difference tests were conducted based on Dawson and Richter’s (2006) 
work. The Bonferroni correction was applied to maintain a family-wise alpha level of .05 
through four slope differences tests.“ p < .013 
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Conclusion 


A large cohort of students was studied in this 
four-year longitudinal study. The contribution 
of this study was to explain how student SEL 
factors, including motivation, social 
engagement, and self-regulation, interact with 
middle school academic achievement and 
gender when predicting later academic 
achievement in high school. Two research 
questions were examined: (1) Do male and 
female students in grades 7-9 differ on SEL 
characteristics? (2) Do students’ SEL factors 
interact with gender and prior academic 
achievement in the prediction of achievement 
in grades 11-12? The results showed that 
females scored higher than males for 
motivation and self-regulation. Further, all 
three SEL factors showed slight incremental 
contribution (less than 1%) beyond gender 
and prior achievement. In predicting later 
academic achievement, positive effects of 
motivation and social engagement were 
observed for students with higher prior 
achievement, regardless of gender. However, 
positive effects of self-regulation were only 
observed for female students with higher prior 
achievement. Generally, the Matthew Effect 
was evident for all three SEL factors. In 
particular, the effect was found with gender 
differences being present in self-regulation. 


The students’ SEL skills in this study were 
assessed through self-ratings, which have 
known measurement issues such as self- 
rating biases, social desirability, and others 
(Kyllonen, 2005; Paulhus, 1984). In addition, 
Paulhus (1984) argued that an individual 
might engage in self-deception or impression 
management when responding to self-report 
items. Individuals engaging in self-deception 
actually believe their positive self-reports. 
Individuals engaging in impression 
management respond to items in ways that 


would present themselves positively to 
others. In this study, it is possible that low- 
achieving male students may have rated 
themselves highly on SEL factors (i.e., 
motivation and social engagement) due to 
either self-deception or impression 
management. 


An alternative explanation for possible biases 
is that individuals respond to items by 
comparing their SEL skills to their peers with 
similarly low achievement (Kyllonen, 2005). In 
this context, low-achieving students might 
perceive “high” SEL factors. This is why 
psychosocial assessments are generally 
considered to be low-stakes and are seen as 
less suitable as the sole criterion for selection 
and other high-stakes purposes. Using fake- 
resistant psychosocial assessments (e.g., 
situational judgment tests), others’ ratings 
(e.g., collecting objective behavioral data 
from institutional records, teachers, or 
counselors), and forced choice methods have 
been documented in the literature as feasible 
ways to minimize the aforementioned 
measurement problems and biases. 


Implications 


Snow, Corno, and Jackson (1996) argued 
that affective (e.g., emotion), conative (e.g., 
motivation), and cognitive individual 
differences should be considered 
simultaneously when predicting students’ 
scholastic performance and vice versa. 
Inspired by Snow’s legacy on the cognitive- 
affective-conative triad (a.k.a., aptitude 
complexes) for underscoring the interaction 
among the complexes to explain student 
learning outcomes (Corno, et al., 2002), 
current findings suggest that students’ SEL 
factors only demonstrate positive influences 
on high school academic performance if 
students have built a foundational knowledge 
base in middle school. In other words, low- 
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achieving students in middle school have a 
lower likelihood of achieving college 
readiness regardless of their levels of SEL 
factors. Therefore, teachers should recognize 
that SEL factors come into play in increasing 
students’ academic achievement in high 
school only after they have achieved a 
foundational level of knowledge and skills in 
middle school. 


The students in this study exhibited a wide 
range of achievement and SEL scores 
suggesting that differentiated interventions in 
the classroom are needed to meet their 
individual needs. For example, a meta- 
analysis that examined universal school- 
based social-emotional development 
programs (Durlak, Weissberg, Dymnicki, 
Taylor, & Schellinger, 2011) found that SEL 
interventions that include programming that is 
Sequenced (programs with a connected and 
coordinated set of activities), Active (active 
forms of learning), Focused (programs having 
components to develop personal or social 
skills), and Explicit (programs targeting on 
specific SEL skills) result in multiple positive 
outcomes in terms of SEL skills, attitude, 
emotions, behaviors, and subsequent 
academic performance. Similarly, evidence 
from response to intervention (RTI) programs 
suggests that SEL programming at each tier 
(universal, targeted, or intensive) results in 
positive behavior changes and improved 
academic outcomes. Examples of these 
programs include Check & Connect, Check- 
In/Check-Out, Safe & Civil Schools 
Foundations, and many others (Hoyle, 
Marshall, & Yell, 2011; Maggin, Zurheide, 
Pickett, & Baillie, 2015; Medigan, Cross, 
Smolkowski, & Strycker, 2016). Durlak et al. 
(2011) suggested that students’ performance 


on SEL can serve as references for 
educators to design further educational 
interventions. 


Finally, when referring to educational 
accountability, policymakers have recently 
recognized that both cognitive and SEL skills 
matter. This study adds to the growing 
discussion on how to leverage efforts to 
develop accountability systems that integrate 
SEL factors, such as the development of 
standards, assessments, and curriculum 
focused on SEL, as well as a more holistic 
approach that integrates core academic 
knowledge and skills, SEL/behavioral skills, 
and education and career navigation in K-12 
programs. 


Future Research 


Examining narrow SEL characteristics is 
suggested as a next step. It is hypothesized 
that Academic Discipline may play a more 
prominent role in predicting later academic 
achievement, whereas the other two SEL 
characteristics that comprised the motivation 
construct in this study (Commitment to 
School and Optimism) may have a smaller 
contribution. 


We pursued gender differences because this 
finding has been seen in the literature before. 
However, as we focus on gaining a clearer 
understanding of how to close achievement 
gaps, we need to examine other potential 
moderators including: race/ethnicity, 
socioeconomic status levels, first-generation 
college student status, and other 
characteristics that are implicated in the 
literature in relation to achievement gaps. 
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